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Abstract 


A  task  centric  approach  to  interface  design  entails  an  explicit  representation,  of 
actions  -  tasks  -  that  need  to  be  performed  by  the  operator.  The  interface  may  represent 
tasks  in  the  form  of  icons  on  a  display  screen  that  the  system  has  determined  actionable 
given  the  current  tactical  information  and  Rules  of  Engagement  (ROE).  The 
representation  of  work  in  tenns  of  a  task  serves  as  a  trace  in  the  system  that  enables 
designers  to  track  workload  in  addition  to  the  task  progress  and  flow  of  tasks  among  team 
members.  Using  Queueing  Theory  statistics,  performance  for  two  Air  Defense  Warfare 
Teams  were  analyzed.  This  analysis  revealed  that  task  allocation,  work-  flow  and  the 
internal  dynamics  of  the  two  teams  were  very  different.  Interestingly,  neither  team 
allocated  tasks  to  team  members  as  envisioned  by  the  system  designers.  Bottlenecks, 
unforeseen  by  the  system  designers,  had  been  introduced  by  the  dynamics  of  the  team. 
These  bottlenecks  were  more  pronounced  for  one  of  the  teams  and  led  to  quantifiable 
differences  in  the  queuing  statistics.  In  particular,  substantial  differences  in  the  average 
life  of  a  task  and  average  number  of  outstanding  tasks  operators  had  to  perform  were 
observed. 

Introduction 

In  future  Combat  Weapon  Systems  (CWS)  three  important  system  design  trends 
are:  1)  focus  on  increasing  capability  while  retaining  constant  or  reduced  manning  levels, 
2)  increased  system  automatic  functions  in  a  Total  Ship  Computing  Environment 
(TSCE),  3)  expected  use  of  re-configurable,  collaborative  task  teams.  Automation  will 
assist  in  meeting  these  design  requirements;  however,  with  increased  automation  combat 
systems  require  human  supervision  and  control.  The  change  from  manual  control  to 
automatic  monitoring  alters  the  role  of  the  human  operator  from  that  of  controller  to 
supervisor  in  CWS.  This  change  of  roles  generates  design  requirements  for  systems  that 
provide  infonnation  tailored  for  supervisory  control  tasks.  For  team  leaders  and  decision¬ 
makers,  this  requires  monitoring  of  both  humans  and  automation  in  distributed  teams, 
across  multiple  mission  phases.  The  collection  of  system  support  services  includes  an 
array  of  computer  technologies  in  a  collective  system,  we  have  termed  “Intelligent 
Mission  Management  &  Monitoring”  (IM3).  These  services  include  both  individual  and 
team  task  and  workload  management,  with  decision  support  for  both  critical  thinking  and 
naturalistic  decision  styles. 

Future  multi-tasking  environments  will  require  that  supervisory  tasks  shall  span 
mission  planning,  rehearsal,  execution  and  assessment  phases  of  missions.  Optimized 
manning  implies  concurrent  activities  that  would  otherwise  be  performed  by  multiple 
individuals  in  today’s  ship  can  instead  be  performed  through  a  multi-tasking  process. 
This  complex,  multi-task  environment  shall  require  operators  to:  maintain  an  awareness 
of  overall  system  functioning,  prioritize  tasks,  and  allocate  resources  in  ways  consistent 
with  their  new  roles  and  new  objectives.  An  understanding  of  these  unique  needs  will 
become  a  critical  element  for  success  in  key  mission  areas  such  as  joint  land-attack 
warfare  in  a  littoral  environment.  The  problem  for  designers  is  defining  the  information 
requirements,  display  fonnats,  and  controls  needed  to  support  adequate  task  performance 


for  these  newer  supervisory  control  activities.  What  is  needed  is  an  understanding  of 
these  unique  needs  that  can  readily  be  translated  into  system  design  requirements. 

The  goal  of  our  research  is  to  develop  quantitative  models  of  Operator  and 
System  performance  that  will  form  the  basis  of  a  scientific  design  approach  that  can  be 
utilized  by  future  Combat  System  Design  Engineers.  In  general,  the  purpose  of  modeling 
is  to:  1)  Predict  impact  of  design  on  human  performance  before  the  system  is  built;  2) 
Compare  alternative  designs;  3)  Compare  alternative  job  structures,  positions,  team 
definitions;  4)  Predict  and  compare  performance  results  for  design  reference  missions; 
5)  Reduce  design  risk;  6)  Identify  design  changes  and  corrections  before  costly  mistakes 
are  made. 

These  models  are  being  developed  for  Air  Defense  and  Land  Attack  combat 
systems,  and  are  being  incorporated  into  prototypes  of  the  future  Multimodal 
Watchstation  (MMWS)  and  Land  Attack  Combat  System  (LACS).  These  models  will 
guide  system  design  in  an  efficient  manner,  contributing  to  a  scientifically  supported 
design  process.  Several  projects  (e.  g.,  MMWS,  LACS,  and  Combat  Supervisory  Support 
Systems)  have  demonstrated  tools  that  fonn  the  foundation  for  further  development  of 
interface  concepts  that  will  enable  operators  to  plan  and  execute  complex  tasks  within 
dynamic  and  multiple  warfare  areas.  There  is  a  growing  need  to  model  these  interface 
concepts  so  that  future  interface  designs  may  evolve  in  a  principled  and  systematic 
fashion.  The  payoff  would  be  the  creation  of  a  "true  engineering  method  for  interface 
design"  (Kieras,  2002). 

Approach 

In  order  to  explicitly  present  the  multitasking  within  a  system,  a  Task  Manager 
(TM)  Display  was  incorporated  into  the  design  of  the  MMWS  and  LACS.  The  TM 
Display  represents  tasks,  in  the  form  of  icons  on  a  display  screen,  that  the  system  has 
determined  actionable  given  the  current  tactical  information  and  Rules  of  Engagement 
(ROE).  As  a  reflection  of  the  type  and  amount  of  work  to-be-done,  the  TM  display 
provides  a  basis  for  attention  allocation  for  each  operator  and  task  allocation  among  a 
team  of  operators.  Tasks  represented  may  range  from  information  updates  to  important 
tactical  responses.  This  supports  human  cognitive  functions  relating  to  work  planning, 
management,  strategy,  projection  and  task  allocation.  Thus,  the  TM  display  embodies  a 
task  centric  approach  to  design  in  that  the  interface  explicitly  represents,  to  the  operator, 
what  actions  (tasks)  need  to  be  performed.  For  further  details  of  the  MMWS  and  Task 
Manger  Display  see  Osga,  Van  Orden,  Campbell,  Kellmeyer,  and  Lulu  (2002). 

The  representation  of  work  in  terms  of  a  task  serves  as  a  trace  in  the  system  that 
enables  designers  to  track  workload  in  addition  to  the  progress  and  flow  of  tasks  among 
team  members.  The  posting  of  tasks  to  the  TM  display  for  operators  to  perform  is  also 
analogous  to  service  calls  arriving  at  a  Help  Desk.  Quantitative  models  and  methods  to 
analyze  system  performance  have  been  developed  for  these  systems  in  the  domain  of 
Queuing  Theory  (Kleinrock,  1975).  We  have  modeled  the  team  of  4  MMWS  operators 
from  the  point  of  view  of  a  Queuing  Network.  A  queueing  network  (Hock,  1996)is  a 


collection  of  queueing  systems  connected  to  each  other  in  certain  ways.  Each  queueing 
system  in  its  general  form  consists  of: 

A)  The  input  or  arrival  process  is  usually  modeled  as  a  stochastic  process,  such  as 
a  Poisson  process  in  which  the  arrival  rate  is  simply  the  reciprocal  of  the  mean 
inter-  arrival  time  of  customers.  In  our  case,  the  customers  are  tasks  arriving 
on  the  TM  display. 

B)  The  service  mechanism  refers  to  the  number  of  "servers"  and  the  lengths  of 
time  the  customers  hold  servers.  This  is  usually  modeled  with  a  negative 
exponential,  and  in  our  case  this  is  the  number  of  operators  and  the 
distributions  of  reaction  times  it  takes  operators  to  perform  various  tasks. 

C)  The  queueing  policy  entails  the  method  by  which  the  system  selects  customers: 
first-come-first-served  (FCFS),  last-come-first-served  (FCFS),  by  priority,  or 
at  random. 

The  relationships  among  the  combat  system  team  members  and  the  manner  in 
which  they  must  process  tasks  may  be  modeled  as  a  network  of  interactive  queues.  For 
example  in  an  "open"  queuing  system,  "customers"  (contacts)  are  processed  and  passed 
from  one  server  to  another.  Thus  the  customer  sequentially  arrives  at  different  queues,  is 
waited  upon  by  a  server  (and  sometimes  must  "feedback"  and  return  to  a  previous  server) 
and  eventually  leaves  the  system. 

In  Figure  1  we  have  diagrammed  an  open  queuing  network  for  the  Air  Warfare 
Coordinator  (AWC),  Air  Intercept  Coordinator  (AIC)  and  Information  Quality 
Coordinator  One  (IQC1)  -  all  of  whom  are  operators  of  the  MMWS  Air  Defense  team. 


V 


Figure  1.  General  Open  Network  Queueing  Model.  Circles  represent 
operators.  Arrows  represent  tasks  that  flow  into  and  out  of  the  system. 
Operators  also  pass  tasks  among  themselves. 


Results 


Queueing  Theory  provides  a  set  of  global  statistics  that  allows  one  to  analyze  the 
overall  system  performance  as  well  as  the  performance  of  any  one  server.  For  example, 
the  average  number  of  tasks  each  operator  has  to  perform,  and  the  average  total  number 
of  tasks  that  are  in  the  system  may  be  detennined.  Other  valuable  performance  statistics 
include:  1)  The  average  life  of  a  task  -  from  its  birth  (arrival  to  the  system)  to  its  death 
(task  completion),  and  2)  Waiting  time  -  the  average  time  a  task  must  wait  before  it  is 
started.  Another  characterization  of  a  queueing  system  is  to  determine  the  percentage  of 
time  the  system  spends  in  various  “states”.  A  state  refers  to  the  total  number  of 
customers  that  are  in  the  queue  at  any  one  time  (those  customers  that  are  either  being 
served  or  waiting  for  service).  In  our  application,  a  state  represents  how  many  tasks  an 
operator  had  to  perform  on  the  Task  Manager  display.  Over  the  course  of  testing  a  system 
and  a  team  of  operators  on  a  scenario,  the  total  amount  of  time  the  system  spends  in  any 
one  state  is  a  measure  of  workload  management  since  this  represents  the  degree  to  which 
operators  kept  pace  with  tasks  appearing  on  the  TM  display. 

Data  from  an  ADW  team  consisting  of  four  operators  (AWC,  IQC1,  AIC,  and 
Tactical  Action  Officer  -  TAO)  were  collected  from  an  one  hour  and  thirty  minute  ADW 
scenario  entitled  the  Sea  of  Japan  (SOJ).  Data  from  this  scenario  were  analyzed  from  the 
viewpoint  of  queueing  theory.  Analysis  of  two  ADW  teams  revealed  two  very  different 
patterns  of  workload  management. 

First,  we  demonstrated  that  a  shift  from  periods  of  low  to  high  workload 
corresponded  to  a  shift  in  states.  For  example,  during  the  low  workload  interval,  for 
nearly  60%  of  the  time,  the  Team  1  AWC  had  no  outstanding  tasks  to  perfonn  on  the  TM 
display.  This  percentage  drops  sharply  to  roughly  5%  during  the  high  workload  period 
(see  Figure  2).  This  same  shift  can  be  seen  in  the  data  for  the  AWC  of  Team  2  (see 
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Figure  2.  System  states  or  percentage  of  time  the  AWC  of  Team  1  had  0-12 
tasks  to  perform  -  Interval  1  low  workload.  Interval  2  high  workload. 


Figure  3),  only  now  the  shift  is  more  pronounced  -  for  50%  of  the  time  this  AWC  had  7- 
12  outstanding  tasks  to  perform.  In  addition  to  this  backlog  during  the  high  workload 
period,  the  data  revealed  that  the  Team  2  AWC  generally  had  more  outstanding  tasks 
during  the  low  workload  interval  than  the  Team  1  AWC.  This  plot  is  supported  by 
comparing  the  average  task  life  (Figure  4)  and  the  average  number  of  tasks  outstanding 
(Figure  5)  for  the  Team  1  and  Team  2  AWCs.  In  Figure  4  and  5  the  Intervals  1  and  4 
correspond  to  relatively  low  workload  periods  where  as  Intervals  2  and  3  correspond  to 
high  workload  periods.  Across  all  intervals,  the  average  number  of  tasks  and  task  life 
was  considerably  greater  for  the  Team  2  AWC  than  the  Team  1  AWC. 
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Figure  3.  System  states  or  percentage  of  time  the  AWC  of  Team  2  had  0-12 
tasks  to  perform  -  Interval  1  low  workload,  Interval  2  high  workload.  AWC  is 
falling  behind  on  completing  tasks. 
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Figure  4.  Average  Task  life  for  AWC  of  Teams  1  and  2.  Task  life  is 
much  longer  for  Team  2  than  1. 


Figure  5.  The  Average  number  of  tasks  on  the  TM  display  for  the  AWC  of 
Team  1  and  Team  2.  In  general,  the  Team  2  AWC  has  more  outstanding 
tasks  to  complete. 


The  immediate  question  was,  why  did  the  AWC  of  Team  2  fall  behind  to  a  greater 
degree  than  the  Team  1  AWC?  One  might  suspect  that  the  AWC  of  Team  2  simply  took 
longer  to  perform  tasks;  however,  analysis  of  keystroke  data  showed  that  this  was  not  the 
case.  To  answer  this  question,  video  recording  of  these  teams  taken  during  the 
experiment  were  analyzed.  This  analysis  revealed  that  task  allocation,  work-  flow  and  the 
internal  dynamics  of  the  two  teams  were  very  different.  Interestingly,  neither  team 
allocated  tasks  to  team  members  as  envisioned  by  the  system  designers.  In  Figures  6  we 
depict  the  different  task  flows  between  an  ideal  team  (that  envisioned  by  the  system 
designers)  and  Teams  1  and  2.  These  figures  represent  different  networks  of  queues  and 
each  is  a  variation  of  the  network  of  queues  depicted  in  Figure  1 . 

The  main  difference  between  the  ideal  team  and  the  tested  teams  is  the  degree  to 
which  the  team  members  passed  tasks  between  themselves.  The  ideal  team  members 
handle  tasks  independently  and  in  parallel;  however,  for  teams  1  and  2  there  were  various 
degrees  of  “meddling”  between  team  members  when  it  came  to  specific  tasks.  For 
example,  for  both  teams,  Queries  and  Warnings  were  never  issued  by  the  IQC1  unless  the 
AWC  specifically  ordered  the  IQC1  do  so.  In  order  to  give  this  order  the  AWC  had  to 
spend  some  time  evaluating  the  task,  hence  the  load  of  tasking  for  the  AWCs  was  greater 
than  that  envisioned  for  the  ideal  team  which  allowed  the  IQC1  to  handle  Queries  and 
warnings  autonomously. 
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Figure  6.  Analyzed  work  flow  data  from  SOJ  Scenario.  Task  allocation 
and  internal  team  dynamics  were  different  for  the  two  teams.  Neither  team 
allocated  tasks  to  team  members  as  envisioned  by  system  designers. 


In  Figure  7,  we  depict  one  of  three  cases  that  describes  the  interaction  between  the  AWC 
and  IQC1  in  order  to  perform  the  query  task.  In  this  case,  IQC1  has  finished  his 
evaluation  but  cannot  send  the  query  because  he  has  not  yet  received  an  order  to  do  so. 
The  IQC1  begins  another  task  and  at  some  later  point,  when  ordered  by  the  AWC,  returns 
to  this  task  to  send  the  query  and  listen  for  a  response.  Thus  bottlenecks,  unforeseen  by 
the  system  designers  had  been  introduced  by  the  dynamics  of  the  team.  These 
bottlenecks  were  even  more  pronounced  for  Team  2  because  in  addition  to  Query  and 
Warning  bottlenecks,  Team  2  imposed  bottlenecks  for  New  and  Update  Track  reports. 
The  IQC 1  of  team  2  was  required  to  read  these  reports  aloud  to  the  other  team  members 
before  the  AWC  sent  off  the  report.  Thus  in  the  video,  one  actually  sees  the  AWC’s 
finger  poised  over  the  send  button  of  a  new  track  report,  as  he  waits  for  the  IQC1  to 
complete  the  reading  aloud  of  the  message. 
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Figure  7.  Interaction  between  AWC  and  IQC1  in  order  to  perform  the  task 
of  Querying  an  aircraft.  The  IQC  1  has  completed  his  evaluation  but  waits  to 
perform  the  query  until  specifically  ordered  to  do  so  by  the  AWC. 


Further  evidence  for  Team  2’s  collaborative  handling  of  new  track  reports  can  be 
found  in  the  data.  For  example,  Team  2  took  36  %  longer  to  complete  New  Track  reports 
and  sent  21%  fewer  New  Track  reports  than  Team  1.  In  addition,  the  number  of  New 
Track  tasks  that  were  selected  solely  by  the  Team  2  AWC,  was  one  third  that  of  Team  1. 
Selection  of  a  task  by  only  one  operator  suggests  that  the  team  did  not  collaborate  on  that 
task. 


To  conclude,  bottlenecks,  unforeseen  by  the  system  designers,  had  been 
introduced  by  the  dynamics  of  the  team.  These  bottlenecks  were  even  more  pronounced 
for  Team  2  because  in  general,  Team  2  made  task  completion  a  collaborative  effort, 
hence  the  slower  task  through-put.  Collaborative  task  sharing  raises  the  broad  and 
important  issue  of  team  situational  awareness  that  must  be  further  explored. 

Conclusion 

Our  research  approach  supports  model-based  design  as  opposed  to  creative 
engineering.  We  believe  that  the  latter  approach  lacks  the  ability  to  predict  human 
performance.  Performance  predictability  is  essential  to  good  design.  In  addition  to 
providing  a  set  of  useful  global  statistics  that  may  be  used  to  analyze  team  and  system 
performance,  Queuing  theory  provides  fonnulas  -  quantitative  predictions  for  these 
statistics.  These  formulas  are  based  on  assumptions  of  input  and  output  task  flow  and 
task  prioritization.  We  are  currently  developing  a  predictive  model  for  the  ADW  team 
viewed  as  a  queueing  network.  Our  ultimate  goal  is  to  use  these  fonnulas  to  predict  and 
evaluate  operator  and  system  performance.  These  quantitative  models  may  then  be  used 
to  simulate  and  quantify  the  effects  of  increasing  and  decreasing  team  size  and  will 
provide  a  model  of  manning  and  automation  requirements.  The  nature  of  task  allocation 
and  dynamic  task  reallocation  schemes  among  team  members  and  autonomous  agents 
may  be  tested  with  these  models.  Of  course  in  all  these  cases,  predictions  need  to  be 
verified  against  actual  teams  of  operators  and  automation.  However,  the  quantitative 
nature  of  these  models  will  guide  the  data  collection  by  suggesting  where  designers 
should  look  for  significant  design  improvements.  Thus  these  models  will  guide  system 
design  in  an  efficient  manner,  creating  a  science  of  design. 
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Decision  Support  Systems  and  Models  for 
_ Intelligent  Mission  Management _ 


Background 

•Multi-mission,  multi-tasking,  optimally 
manned  CICs  will  require  greater 
reliance  on  automation. 

•Operators  will  require  resource 
management  tools  and  planning  aids  to 
meet  mission  requirements  -  these  must 
reduce  workload  in  the  planning  and 
execution  process 


GOALS 

1 .  Model  individual  operator  and  team 
performance. 

2.  Simulate  and  quantify  the  effects  of 
increasing  and  decreasing  team  size 
providing  a  model  of  manning  and 
automation  requirements. 

3.  Test  the  nature  of  task  allocation  and 
dynamic  task  reallocation  schemes  among 
team  members  and  autonomous  agents. 

4.  Develop  methods  to  dynamically  predict 
team  performance. 

5.  Develop  displays  to  depict  actual  team 
performance  dynamically  to  team  leaders 
and  methods  to  recommend  changes 
towards  optimization. 

6.  Discover  behavioral  results  of  team 
performance  awareness  with  regard  to 
team  self-monitoring  and  correction. 
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•  Predict  impact  of  design  on  human  performance  -  before 
system  is  built. 

•  Compare  alternative  designs. 

•  Compare  alternative  job  structures,  positions,  team 
definitions. 

•  Predict  and  compare  performance  results  for  design 
reference  missions. 

•  Reduce  design  risk. 

•  Identify  design  changes  and  corrections  before  costly 
mistakes  made. 
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“Air  Defense  Battle  Group  Task  Monitoring 


Representation  of  work  in  terms  of  tasks  servers  as  a  trace  - 
enables  designers  to  track  workload  and  flow  of  tasks  among  team 
members. 

Posting  of  Task  analogous  to  customers  arriving  at  a  queue  for 
service:  Model  Teams  with  Queueing  Theory  and  Queueing 
Networks. 
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General  Network  Queueing 
Model  of  AWC  Team  . 
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Components  of  Queueing  Model 

A)  The  input  or  arrival  process  is  usually  modeled  as  a 
stochastic  process,  such  as  a  Poisson  process.  In  our 
case,  the  customers  are  tasks  arriving  on  the  TM 
display. 

B)  The  service  mechanism  refers  to  the  number  of 
"servers"  and  the  lengths  of  time  the  customers  hold 
servers.  This  is  usually  modeled  with  a  negative 
exponential,  and  in  our  case  this  is  the  number  of 
operators  and  the  distributions  of  reaction  times  it 
takes  operators  to  perform  various  tasks. 

C)  The  queueing  policy  entails  the  method  by  which  the 
system  selects  customers:  first-come-first-served 
(FCFS),  last-come-first-served  (LCFS),  by  priority, 
or  at  random. 


Queueing  Models 

Different  Team  Task  Allocation  Schemes  can  be 
represented  by  Network  Queueing  Models. 

Queueing  Models  make  Quantitative  Predictions  of 
“Throughput”  different  for  each  team: 

1)  Average  Time  a  Task  Stays  in  the  System 
(from  “birth  to  death”). 

2)  Average  Number  of  Tasks  in  the  System. 

3)  Average  Number  of  Tasks  for  each  operator. 

4)  The  distribution  of  system  states  -  the 
percentage  of  time  there  are  n  outstanding 
tasks  to  perform. 
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Air  Def.  Warfare  MMWS  Experiments 


•  Four  5 -member  ADW  teams  were  tested  on  a  2  hour  Scenario  -  Sea  of  Japan  (SOJ). 

•  Tactical  Action  Officer,  Air  Warfare  Coordinator,  Information  Quality  Control  (2), 
Air  Intercept  Controller. 

•  Operators  were  assigned  Primary  and  Secondary  Tasks. 

•  All  system  recommended  tasks  were  presented  on  a  Task  Manager  (TM)  Display. 

•  All  Teams  “self-organized”  -  were  “free”  to  allocate  tasks  amongst  themselves  -  not 
told  how  or  when  to  reallocate. 

•  Only  support  for  allocation  was  visual  -  listing  of  tasks  on  the  TM  display. 

The  results  provide  a  basis  for  building  team  models. 

Results  show  a  contrast  between  team  performance  outcomes. 
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Team  1  Air  War  Coordinator  “States” 


%  of  Time  spent  by  number  of  tasks  to  perform. 


Interval  1  vs.  Interval  2 
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Scenario  Interval  1  (Low  workload)  Scenario  Interval  2  (High  workload). 

Higher  workload  interval  marks  a  shift  in  states  where  AWC  begins  to  fall  behind  completing  tasks. 
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Team  2  Air  War  Coordinator  “States” 

%  of  Time  spent  by  number  of  tasks  to  perform. 

Interval  1  vs.  Interval  2 
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Scenario  Interval  1  (Low  workload)  Scenario  Interval  2  (High  workload). 

Higher  workload  interval  marks  a  shift  in  states  where  AWC  begins  to  fall  behind  completing  tasks. 
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Team  1  and  2  AWC  States  Comparison 
During  Period  of  High  workload 

%  of  Time  spent  by  number  of  tasks  to  perform. 

Interval  2 
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Scenario  Interval  1  (Low  workload)  Scenario  Interval  2  (High  workload). 

Higher  workload  interval  marks  a  shift  in  states  where  AWC  begins  to  fall  behind  completing  tasks. 


SPAWAR 
Systems  Center 
San  Diego 


Team  1  and  2  AWC  Average  Number  of 
Tasks  During  Low  and  High  Work  Load 

Intervals 

Average  Number  of  Tasks  on  TM  Display 


Workload  Intervals 
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Team  1  and  2  AWC  Average  Task  Life 
During  Low  and  High  Work  Load  Intervals 
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Team  Task  Allocation 


AWC  for  Team  2  falls  behind  in  his  work  to  a 
much  greater  extent  than  the  AWC  for  Team  1. 

Why?  -  Analysis  of  T earns  revealed  that  "ask 
Allocation  and  Work  Flow  was  different  between 
these  two  teams. 


Goal:  Modeled  these  difference  with  : 

-  Network  Queueing  Theory 
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Network  Queueing  Model  of  “Ideal  Team”  - 
Envisioned  by  System  Developers 
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Network  Queueing 
Model  of  Team  1 
Task  Flow. 
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What  you  are  going  to  see: 

9:33  Level  1  track  1035  arrives 

9:44  Hear  voice  of  supervisor  informing 
AWC  about  the  Level  1  task. 

9:45  AWC  selects  task 

AWC  gets  side-tracked  listening  to  Bravo 
Whiskey 

New  tasks  begin  to  pile  up. 

10:13  AWC  orders  IQC1  to  send  Level  1. 

10:18  IQC1  acknowledges  order. 

What  you  don’t  see  but  we  know  from 
data  log 

IQC1  selects  task  only  after  AWC  orders 
him  to  send. 


IQC1  selects  task  at  10:38  and  sends 
Level  1  at  10:41. 
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Flow  of  a  level  1  query  task  -  AWC  and  IQC1  are  both 
involved  in  task  -  AWC  makes  decision  to  send 
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Network  Queueing  Model  of 
Team  2  Task  Flow  Alternative. 
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What  you  are  going  to  see: 

6:00  New  Track  (NT)  1043  (commair) 
arrives 

6:16  New  track  1010  (unknown) 
arrives. 

6:16  IQC1  announces  NT  1043 

6:19  AWC  selects  1043  -  does  not 
finish  task. 

6:21  IQC1  announces  NT  1010 

6:26  AWC  selects  NT  1010 

6:37  AWC  sends  NT  report  for  track 
1010 

6:40  AWC  reselects  NT  1043. 

6:52  AWC  sends  NT  report  for  track 
1043 
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Evidence  that  Team  2  Handled  New  Track  reports 

Differently  than  Team  1 


Team  2  takes  36%  longer  to  complete  New  Track  Reports 


Systems  Center 

Evidence  that  Team  2  Handled  New  Track  reports 

Differently  than  Team  1 

Team  2  completes  21%  fewer  New  Track  Reports  than  Team  1 
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Evidence  that  Team  2  Handled  New  Track  reports 

Differently  than  Team  1 


Team  2:  More  operators  are  involved  with  the  New  Track 
Report  Task  for  Team  2.  AWC  rarely  is  the  only  operator 
to  select  a  New  Track  Report  Task. 
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Conclusions 

Performance  on  New  Track  Report  Tasks  was  very  different 
for  the  two  teams  because: 

1)  The  Teams  chose  to  handle  the  tasks  very  differently. 

2)  Team  2  created  sub-tasks  that  involved  more  than  one 
operator. 

3)  Making  the  task  a  collaborative  effort  among  team 
members  may  have  increased  Situation  Awareness  but 
this  came  at  a  price: 

-  Increased  Queue  Time  for  the  New  Track  Report  Task. 


General  Conclusions 

Model-based  Design  Provides  Performance  Predictability 
which  is  essential  to  good  design. 

Current  Goals: 

1 .  Develop  a  predictive  model  for  the  Air  Defense  Warfare 
team  viewed  as  a  queueing  network. 

2.  Evaluate  operator,  team  and  system  performance  with 
these  models. 

3 .  Explore  the  nature  of  task  allocation  and  dynamic  task 
reallocation  among  team  members  with  these  models. 
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General  Open  Network 
Queueing  Model 
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Network  Stats: 


Load  to  each  node: 


p'=  1 


Ave  #,  N,  of  tasks  in 
the  whole  system: 


N=  2  \  /(Mi  -  K) 


Ave  time,  T,  of  tasks 
in  the  system: 


T=  J_E  k,  /(Pi  -  A,) 


■Yr 


Probability  of  a  particular 
state  (nl5  n2,  n3)  tasks: 

P(n)=n(l-Pi)  p  m 


^1 _ 
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Queueing  Theory 


•  Because  the  teams  handled  the  tasks  differently,  the  p,;, 
and  p;  are  different  for  the  different  teams,  so  formulas  for 
average  number  of  task,  average  time  spent  in  the  system, 
average  time  spent  waiting  should  yield  different  results  for 
the  different  teams. 


•  One  critical  aspect  of  our  operators  was  not  captured  by 
these  models.  Normally  when  no  tasks  are  present,  the 
server  is  idle;  however,  this  was  not  the  case  with  human 
operators.  When  there  were  no  tasks  on  the  TM  displays, 
operators  examined  the  TACSIT  display. 


These  non  -TM  tasks  must  be  taken  into  account  in  order  to 
quantify  system  performance  because  they  will  have  an 
impact  on  the  queueing  statistics. 
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Queueing  Theory 


•  A  queue  with  “service  vacations”  can  be  adapted  to  model 
our  situation  (Takagi,  1991). 

•  If  there  are  no  customers  in  the  queue  that  need  to  be  served, 
the  server  takes  a  vacation. 

•  If  the  operator  has  no  tasks  on  the  TM  display  he  “takes  a 
vacation”  by  analyzing  information  on  the  TACSIT  display. 
When  he  is  done  looking  at  the  TACSIT  display  he  “returns 
from  vacation”  to  see  if  there  are  any  tasks  on  the  TM 
display. 

•  We  assumed  operator’s  ‘vacation  times’  and  service  times 
were  both  exponentially  distributed  however  the  parameters 
v  and  p  for  vacation  time  and  service  time,  respectively,  are 
not  necessarily  equal.  Arrival  was  assumed  to  be  Possion. 
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Queueing  Theory  Stats  for 
Vacationing  Server 


The  waiting  time  for  a  queue  with  service  vacations  v  is: 


fi  —  A  v 

The  average  time,  T,  a  task  spends  in  the  system  is: 

r  =  _!_+i 

jU  —  A  v 

The  average  Number  of  customers,  N,  in  the  queue: 


N  = 


1  -  p  v 


We  have  extended  this  model  to  include,  task  prioritization. 
Network  formulas  may  also  be  derived. 


